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ABSTRACT 

This report summarizes, evaluates, and synthesizes 
the data on the training value of training devices. The report 
discusses the issues of substitution of some operational training 
time by training devices and the relationship between training 
effectiveness and cost (fidelity of simulation) . These general 
conclusions were made: 1) Experiments reveal that substantial amounts 
of flight time q^n be substituted for by simulator time; 2) Most 
experimental "^tifpiilk has been done on simple aircraft and traineri^; 3) 
Different kind^s oi: flight tasks have different transfer effects; U) 
The levi^l of simulation and kind of trainer importantly influence 
transfer; 5) Careful specification of both trainer and operational 
tasks is necessary if transfer is to occur; 6) Motion of particular 
kinds affects trainee performance and transfer; 7) Addition of motion 
and visual displays increases fidelity requirements; 6) How a device 
is used may influence learning and transfer to a greater degree than 
trainer design; 9) Differences between training and operational 
equipment are necessary to exploit training technology; 10) A precise 
specification of tasks and measures of operational transfer tasks is 
vital to effectiveness evaluation. (JK) 
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ABSTRACT 



ANALYSIS OF THE TRANSFER OF TRAINING, SUBSTITUTION, AND 
FIDELITY OF SIMULATION OF TRAINING EQUIPMENT 

This report sunmiarlzes, evaluates and synthesizes the data on the 
training value of training devices. The report discusses the Issues 
of substitution of some operational training time by training devices 
and the relationship between training effectiveness" and cost (fidelity 
of simulation) . 
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FOREWORD 



This report was prepared as one element a staff study on cost 
and training effectiveness undertaken by a Training Analysis and 
Evaluation Group (TAEG) team. It was prepared by Dr. Gene S. Micheli. 

The main report prepared by the TAEG team, of which this report is 
a supplement, is entitled, "Staff Study on Cost and Training Effectiveness 
of Proposed Training Systems" (NAVTRAEQUIPCEN TAEG Report 1). 

The report is separately published because it addresses an issue 
which has relevance not only for the Staff Study on Cost and Training 
Effectiveness of Proposed Training Systems (NAVTRAEQUIPCEN TAEG Report 1) , 
but also for the broader questions of training equipment (device) fidelity 
of simulation and the substitution of training device-based training for 
training using operational equipments and environments. 
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ANALYSIS OF THE TRANSFER OF TRAINING, SUBSTITUTION, AND 
FIDELITY OF SIMULATION OF TRAINING EQUIPMENT 

"If we could first know where we are, and whither 
we are drifting, we could better judge what to 
do and how to do it." - Abraham Lincoln 

The purpose of this report is to analyze the current situation on 
the cost and training effectiveness of training devices. 

The cost and training effectiveness of training devices are Ideally 
determined by the collection of empirical data by controlled experiments. 
At present, however, there is a paucity of such data. What does exist 
will be summarized in terms of the commonality of findings. Specific 
current training situations will be analyzed to detennine tasks which 
can be learned In the training system and In the operational situation. 

From the results of the analyses of specific transfer/substitution 
studies, an attempt will be made to generalize to various types of 
training situations in order to arrive at an evaluation of the cost and 
training effectiveness of training devices. 

"Cost effectiveness" will be used In this paper to mean the use of 
the least costly of several alternative training systems, all of which 
could equally produce men trained to a specified level of proficiency. 
Lower cost of training equipment allows (even demands) Its use In place 
of operational equipment. 

The "training effectiveness" of a training device is usually 
expressed as a measure of transfer of training. Transfer of training 
refers to the degree to which practice In a trainer carries over to 
(or affects) performance in an operational situation, as compared to the 



performance of trainees who received no practice in the trainer* In 
other words, training effectiveness is the difference between a performance 
measurement on an operational task after practice on the tiraining device 
and performance on the operational task without practice on the training 
device. (Trainees who receive practice in a training device are usually 
referred to as the experimental group; trainees who receive no practice 
in the trainer are referred to as the control group.) 

Thus, transfer of training is the term used to describe how what is 
done (learned) in one situation affects what is done (performance) in 
another. Transfer of training is positive when a training situation 
aids subsequent performance. It is negative when it hinders that 
performance » and it is zero when training has no effect on later 
performance . 

Most measures of training effectiveness are measures of transfer 
of training. Many different formulas exist for expressing the amount 
of transfer (References 1, 2, and 3). 

Percent transfer based on Improvement in performance on the 

operational task or on savings in time to reach a specified performance 

level on the operational task may be calculated from the following formula: 

% transfer = Zc - Ze x 100 
Zc 

Where: 

Zc a performance or time required on the operational (or transfer) 

task by the control group. 
Ze » the corresponding value for the experimental group. 



The Northrop Air Force Future Undergraduate Pilot Training System 
Study (Ref . A) equated the percent transfer formula (based on savings 
in time) with "replacement percent" to denote that in is an index of 
the percent of time on the operational task which can be saved or 
"replaced" by time in the training device. 

Roscoe of the University of Illinois Aviation Research Laboratory 
believed that the quantitative assessment of the transfer of training 
from training devices to operational tasks is not adequately described 
by the percent transfer measure. The fact that trainers result in a 
saving of time to reach a specified level of performance in the opera- 
tional task is meaningful only if the time in the trainer is known. 
This resulted in the development of the transfer effectiveness ratio 
(TER) (Ref. 5). The TER is a measure for assessing the effectiveness 
of a training device by expressing the saving in time on the operational 
task as a function of the amount of time in the trainer. It is defined 
as time saved in the transfer (or operational) task, divided by the 
time required in the training device. The TER may be calculated from 
the following formula: 

TER = Ye - Ye 
Xe 

Where : 

Yc = time required by the control group to reach some criterion of 

proficiency in the operational (or transfer) task. 
Ye = the corresponding value for the experimental group. 
Xe = the training device hours received by the experimental group. 
Examples of calculations of percent transfer and TER follow. 
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From data in Reference 6: 

> 

% Transfer = 60 hrs. - 6.5 hvs . x 100 « 89% 

60 hrs. ^> 

v 

TER = 60 hrs. - 6.5 hrs . = 1.25 ^ 
42.8 hrs. 

From data in Reference 7: ^ 

% Transfer = 211 hrs. - 18 hrs . = 91% (Flight check criterion) 

211 hrs. 1 

TER » 211 hrs. ^ 18 hrs . « .91 (Flight check criterion) 

213 hrs. \ 



% Transfer « 262 hrs. - 113 hrs . = 57% (Criterion of completing 
262 hrs. B-Stage) 

TER 262 hrs. - 113 hrs . = .57 (Criterion of completing B-Stage) 
261 hrs. 

These dtita show that the same training device may exhibit different 
transfer effectiveness ratios depending upon the criterion of performance 
used. Also, for different stages of a curriculum, a training device may 
have different TER*s. And, of course, the effectiveness of a training 
device depends greatly on how it is used. However, although the value 
of measures of effectiveness may change, they are useful for studying 
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learning and transfer. ^ 
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A number of transfer of training experiments have been performed 
which demonstrate that trainers can be used effectively to reduce 
operational (e.g., flight) training time by significant proportions. 
(See Appendix A for suiranaries of training system effectiveness studies.) 

These transfer of training experiments lead to the following 
conclusions : 

•"V 

1. Simulators have cost and training value for pilot training, since J 
they permit the learning of flight tasks in them. In fact, substantial 
cmounts of simulator time can be used in place of flight time. 
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2. Most experimental work has been doue on simple aircraft and 
trainers, but similar results have been obtained when complex aircraft 
and trainers have been used. 

3. Different kinds of flight tasks have different transfer effects. 
Simulators are best for procedural and Instrument flying tasks. Complex 
maneuvers have not been learned as well with the past state-of-the-art 
In simulation. 

4. How a device Is used may Influence learning and transfer to a 
greater degree than trainer design. 

Most of the studies were conducted during the 1940 *s and early 
1950's. Similar research has recently resumed. The University of 
Illinois Aviation Research Laboratory found In 1971 (Ref . 5) that eleven 
hours of training in the old "Blue Box" (AN-T-18) resulted in a savings 
of nine hours of flight time (out of 46 hours) on the Piper Cherokee 
(transfer effectiveness ratio of 0.8). Eleven hours of training in the 
GAT-1 resulted in a savings of eleven hours of flight time (transfer 
effectiveness ratio of 1.0). 

HumRRO, in 1971 (Ref. 6) , conducted an evaluation of the Synthetic: 
Flight Training System (SFTS) , Device 2B24, using Army helicopter 
trainees who had just completed 110 hours of Primary training on the 
TH-55 and were ready to stu^t instrument training (TH-13T). Compared 
to conventionally trained students who spent 60 hours in the aircraft, 
the SFTS trainees spent 42.8 hours in the trainer and 6.5 hours in the 
aircraft. A TER of 1.25 was calculated from the data available in the 
report. Also, calendar time was. eight weeks, versus 12 weeks for the 
conventional piogram. 
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The NAVTRAEQUIPCEN conducted a transfer of training experiment 
in 1971 of Device 2F90, the TA-4J OFT (Ref. 7). The effectiveness of 
the device for training on the basic instrument portion of the advanced 
jet syllabus was evaluated by comparing groups given different training 
regimes. Three experimental groups were compared to each other and to 
a control group which had received the standard syllabus training. Of 
the three experimental groups , one received training only in flight, 
another group only in the trainer » and the third received only academic 
training on related principles of the basic instrument portion of the 
syllabus. All groups were given a flight check in the TA-4J aircraft 
after training. Following the flight check, students were recycled for 
as many flights (in the aircraft or in the trainer) as was estimated 
by the check pilot tri iif^. necessary to make them equivalent to those 
receiving the standii.*;? w.'ilabus. 

The results of the experiment are; 

1. The flight check scores of the control, flight, trainer, and 
academic groups were 3.12, 3.03, 2.99, and 2.77, respectively. The 
control group was best; however, there was no statistically significant 
difference between the flight-trained and the trainer-trained groups. 

2. Even after the students were recycled for as many sessions in 
the aircraft and in the trainer as the check pilots thought they 
required, the trainer groui saved 4.7 flight hours (or three flights) 
compared to the control gro p. However, the trainer group required an 
additional 1.6 hours (or one session) on the trainer. This is a 55% 
savings in flight hours, which translates into a considerable savings 
in cost per trainer group student. 

6 
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3. Calculations for transfer effectiveness ratios resulted in the 
values 0.91 and 0.57, depending upon the definition of criterion 
performance used. The TER's are interpreted to mean that for the 
portion of the syllabus experimented with , trainer sessions are almost 
equivalent to aircraft flights in training effectiveness, or ha^ft an 
equivalent value of 0.57 to 1, depending on whether is used the hours 
required to pass the flight check or the hours to complete the basic 
instrument stage (which includes recycled flights in the aircraft and 
trainer) • 

In an attempt to provide a common basis for comparing the results 
of different studies, percent transfer based on a savings in time 
(replacement percent according to Northrop) and TER's were computed by 
Northrop (Ref. A). The tasks trained were contact flight procedures 
and maneuvers, landing, and takeoff. The percent transfer and Training 
Effectiveness Ratio for various studies are given in Table 1. 

The preceding data convincingly demonstrate that flight simulator 
training transfers to aircraft: and can be substituted for some flight 
time. This is true for civilian pilots of light aircraft, military 
undergraduate pilot training, and airline pilot transition training. 
Further evidence of the transfer and substitutability of flight training, 
of course, is NASA's Apollo Program in which 100% of the training for 
space flight and lunar landings was conducted in simulators. 

Appr\rently it is not true, as is believed by some, that the airlines 
can conduct most (and eventually 100%) of its transition training in 
simulators only because their pilots are very experienced. The studies 
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reported above provide sufficient evidence that flight training devices 
are also effective for neophyte pilot training. 

Some discussion is warranted here about the experience of the 
airlines. They have followed a traditional pattern for many years. 
Each training program was modeled after earlier ones with very little 
change, c; Pilots practiced maneuvers in the airplane to develop skill 
needed for passing a rating check. Accidents occurring in training 
flights while practicing high risk maneuvers and particularly the 
prohibitive cost per flying hour has forced the airlines to use simu- 
lators for transition training. 

The airlines have performed detailed task analyses as a basis for 
defining Specific Behavioral Objectives (SBO's) to restructure their 
training programs to make maximum use of simulators. For example, 
American Airlines has gone from 20.6 hours on its B727 aircraft in 1966, 
to 7.6 hours in 1969 (63% reduction of flight training hours). Check- 
rides by FAA Flight Standards Inspectors have demonstrated that pilots 
trained using a ratio of 28.2 flight hours to 7.6 simulator hours can 
qualify. American Airlines' goal is to achieve 100% training in 
simulators (duo to cost). Its DC-10 transition training program is 
currently two hours of flight time. 

In the military, also, training devices have been used as supplements 
to flight training. For the following reasons, however, there is little 
choice but to substitute for flight time by training devices (Ref. 16): 

Cost: The complexities of current and future aircraft and weapon 
systems are driving the cost per flying hour to such a level that all 
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but "payload" or "mission" flight is prohibitively expensive. On this 
basis, alternatives to present concepts of flight training demand 
investigation, development, and implementation to provide adequately 
skilled aircrews* 

Air Space: The speeds attainable by current and future aircraft 
require greater operating air space per unit than that known in the past* 
This fact, considered with the increasing demands for air service, places 
a premium upon an already overloaded air space. When saturation occurs, 
catastrophe may be the result. The implications for flying safety, as 
well as efficient operations, are readily apparent. Again^ alternatives 
to present practices and procedures must be developed to better use and 
conserve this fixed resource of air space. 

Flying Safety: Each flight in an aircraft is an exposure to danger, 
however small. Training flights, in addition, expose the trainee during 
the period he is least capable of coping with dangerous or emergency 
occurrences. The value of human life is, of course. Incalculable as it 
has always been. The value per unit of current and future aircraft is 
such that the financial penalty for losing an aircraft, when alternatives 
can be made available, is too great to be justified. 

Data such as presented above have apparently convinced Navy and 
Air Force planners that flight substitution is feasible. Consideration 
is being given to the substitution recommendations of the UPT studies. 
The Navy study (North American and Link, Ref . 17) of undergraduate 
pilot training states 45% substitution will be possible overall when 
a wide angle visual for operational flight trainers (OFT^s) becomes 
available. The Air Force UPT studies state that 50% (Northrop, Ref* 4) 
and 47% (Lockheed, Ref. 18) will be possible. 
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The Navy UPT (North American/Link) recommendations for substitution 
of flight time by ground training devices were based on an analysis of 
the tasks involved in flight training* These tasks were analyzed to 
identify the kinds of learning processes involved, and to identify the 
kinds of demand they tend to make on the training setting* It was found 
for example, that most instrument flight trainers are highly procedural, 
and require primarily tnat the training environment contain accurate 
representations of cockpit displays and controls. Other maneuvers , 
aerobatics for example, while containing significant procedural elements 
also require pilot surveillance of a variety of out-of-the-window visual 
cues to position, attitude, heading, and airspeed* Because of the 
relative ease of simulating events represented in cockpit displays, 
motions and sounds, primary attention in allocating training tasks to 
simulation has been given to requirements for the representation of out- 
of-the-window visual information. Each of the non- instrument flight 
maneuvers trained in the undergraduate program requires out-of-the- 
window visual information* Some maneuvers require simple cues which 
are readily provided at reasonable expense* Others require more 
complex cues, at greater cost* A few require visual cues which have 
such extensive equipment implications that their incorporation in ground 
training devices would be uneconomical within the undergraduate program* 
In allocating flight tasks to ground trainers, consideration was given 
to the relative expense of ground and flight training, to assure the 
allocation of tasks to ground training which would, in fact, represent 
significant cost savings. Two hypothetical simulators. Simulator "A" 
and Simulator "B", were conceived in making tentative allocations* 



11 

15 



Training devices using the Siir.ulator "A" concept would have a three- 
degree-of-freedom motion system, and a visual system display CRT would 
have a 48* horizontal and a 28® vertical field of view. Cockpit 
controls and instruments would have the same extent of fidelity in 
current military and commercial flight simulators. Other features, such 
as task and maneuver demonstrations, performance measurement, feedback, 
and permanent recordings of performance, would be included. Training 
devices employing the Simulator "B" concept would have a 180® horizontal 
and 87® vertical field of view and will include a generalized earth 
(or sea), and sky with horizon. It will have a six-degree-of-freedom 
motion system. Other characteristics will be similar to those described 
in the "A" concept. 

A set of ground rules was developed by North American and Link to 
guide the estimation of the relative effectiveness of various allocations 
of flight tasks to ground training devices. The extent of substitution 
of training device time for flight time resulted from the application 
of these ground rulT?.s, In defining ground rules for the reduction of 
flight hours through the use of ground simulation, the recent training 
literature was reviewed for empirical evidence of successful substitutions 
of device for aircraft time. Also, analyses were made of the maneuvers 
to be trained in the recommended program and of the simulator capabilities 
available for supporting this training, to define the pilot task 
elements to which ground training is applicable. Device design concepts 
were developed to incorporate these task element-related capabilities 
based on the identification of task elements within the capabilities 
of device concepts. An analysis of flight tasks likely to be employed 
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in training naval pilots in the future program identified the kinds of 
1^ ^ learning functions involved in each task. This analysis was used to 

anticipate the types of training setting most appropriate to these 
functions, and estimates were made of the extent to which each task 
depends on each function. Guidelines were developed for assigning 
percentages of flight tasks to training devices: 

a. Substitution of simulator for aircraft flight time is determined 
primarily by the proportion of maneuver, or task training time, which 
would be devoted in flight to learning procedural or fixed-sequential 
task elements. Each task involved in aircraft and system operation 
contains a significant procedural component. That is, each task requires 
the selection y initiation and execution of some fixed sequence of 
control outputs whose magnitude rind timing are keyed to sets of relatively 
well-defined events. Ground simulation is particularly effective in 
training these procedural task elements because, by definition, they 
involve cues; to control actions which can be readily Identified and, in 
most cases, adequately represented in simulation. 

b. Approximately 25% of the time normally spent in solo flight can 
be re-allocated to ground simulation. 

c. Approximately 75% of instrument and radio instrument flight 
training, which does not involve an out-of-the-window reference, can be 
provided in ground simulation devices. The motion cue requirements in 
almost all of the procedural and skill elements of instrument flight 
'are well within the state-of-the-art, and only minimal visual requirements 
exist in this training stage. ^ Near-perfect fidelity of instrument 
display and control representation is readily available, making ground 
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simulation almost totally equivalent to actual instrument flight. A 
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The primary discrepancy is in the stress involved in actual aircraft 
flight, and the knowledge that inaccurate performance can have serious 
consequences. For this reason, it is essential that skills learned in 
the unstressed ground trainer environment be demonstrated in actual 

flight. ) 
d. Approximately 50% of dual flight time, not involving motion -v 

% 

and visual capabilities outside the simulation state--of-the-art , can 

be provided in ground simulation. The superior capability of the simu- J| 
lator for permitting instructor monitoring of student performance, the 

- 1 

X. 

capability for practicing maneuvers and procedures which could not be ) 
practiced in the air, and the simulator capability for concentrating 
only on training- re levant task elements contribute significantly to 
its ability to substitute for dual instruction time. 

An example of recommendations made for substitution of simulator 
time that may be made for flight time in the Advanced Jet syllabus is 
shown in Table 2, which is taken from a draft of Ref. 17, dated April 1971. 

If the Navy and Air Force future UPT studies* analyses and conclusions 
that as much as 50% (and 75% in some specific areas) of flight time can ^ 
be substituted for by simulators seem high, consider the viewpoints 
of participants in a symposium on pilot training and the pilot career 
conducted by the Rand Corporation (Ref. 19). 

In discussing the question, "How far do all present want ta'go with 
ground-based simulation in substitution for aircraft?", a representative 
of the University of Illinois Institute of Aviation proposed aiming for 

100%. He projected an answer for the commercial airlines, which would be ^ 
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Table 2. FLIGHT SIMULATOR SUBSTITUTION, CURRENT SYSTEM 

ADVANCED JET, TA-4 



STAGE 


SYLLABUS 
HOURS 


SIM A 
SUB HRS 


SIM A 
SUB % 


SIM B 
SUB HRS 


SIM B 
SUB % 


Familiarization 


11.2 


2.8 


25 


5.6 


50 


Basic Insts. 


8.4 


5.6 


67 


5.6 


67 


Inst. Nav. 


36.1 


29.6 


82 


29.6 


82 


Formation 


12.6 


Q.O 


0 


2.8 


22 


Night Flying 


8.5 


1.5 


18 


5.6 


67 


Oper Nav 


9.3 


0.0 


0 


4.0 


43 


Appl. Nav 


4.5 


3.0 


66 


3.0 


66 


Air/Grnd Wpns 


12.1 


0.0 


0 


5.5 


45 


Tactics 


9.9 


0.0 


0 


4.4 


44 


Air/Air Wpns 


4.0 


0.0 


0 


1.0 


25 


Car Qual 


12.4 


1.6 


13 


4.8 


_39_ 




129.0 


44.1 


34% 


71.9 


56% 

t 
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to take pilots straight from the simulator to flying the aircraft. A 
commercial airlines representative confirmed that for the reason of cost, 
the airlines* objective is to perform 100% training in simulators. 

A representative of the USAF Human Resources Laboratory stated 
that a 100% goal is a meaningful and viable goal provided that constraints 
operating to prevent its achievement are recognized. Such constraints 
are: cost, probability of not achieving 100% fidelity, stress, moti- 
vation, and joy of flying. In his opinion, the 100% goal is not an 

I 

unreasonable aspiration under such conditions. 

Most of the work was done with pilots, probably because of the 
desire for information in such a high risk activity. Another reason 
may be that the feasibility of substitution is readily apparent. Even 
with the paucity of transfer data in other areas, however, there is 
convincing justification for use of training device substitution. Further, 
substitution could be started prior to the collection of empirical 
evidence in many cases because of the relatively small risk involved. 

Studies of non-flying activities in which transfer data was obtained 
follow. Device 3A105, Tracked Vehicle Driving Trainer (M48A3 tank), 
was evaluated at the Tracked Vehicle School, Camp Pendleton (Ref. 20). 
It was found that driver training using the trainer was as effective as 
training on the actual tank* Training was accomplished in approximately 
an equal amount cr time using the trainer versus using the actual tank. 
Since the time differences between those groups using the trainer and 
those using the tank alone are almost equal, it was concluded that there 
was a one-to-one replaceme:nt ratio between the trainer and the tank 
itself as far as driver training is concerned. Further, since there is 
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a significant cost differential between the two modes of training, the 
training device represents a more cost-effective method of training 
tank drivers. 

An evaluation of the Carrier Air Traffic Control Center portion of 
a large Tactical Advanced Combat Direction and Electronic Warfare System 
was conducted at the Fleet Anti-Air Warfare Training Center, Point Loma. 
Transfer data were collected on the USS Constellation and USS Midway 
(Ref. 21). The data indicate that increasing the time spent in the 
trainer results in increased performance at sea. The findings also show 
that team, sub-team, and individual capabilities to deal with recovery 
contingencies and emergencies improve. 

Device 2F69B, P-3A Aircraft Weapon System Trainer, at Patuxent River, 
was evaluated using training squadron ASW crews (Ref. 22). Data 
collected in the trainer indicated an increase in performance throughout 
five Weapon System Trainer (WST) sessions. This increase occurred 
despite the fact that instructor aid was systematically decreased while 
at the same time the level of task difficulty was increased. The 
improvement in task performance was reflected in measures of accuracy 
and efficiency. The accuracy of completing such mission tasks as 
navigational stabilization, search, localization, and attack consistently 
improved during thi! trainer sessions. Improved accuracy was accompanied 
by improved efficiency. As the students progressed through WST training, 
they reduced the time spent in completing each evolution. 

To demonstrate the extent to which training is transferred to the 
airborne enviroiunent , a second phase of the study was directed at 
obtaining performance measures in the operational setting. Attempts 
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to obtain submarine services were unsuccessful; therefore, the transfer 
data portion of the study was conducted using surface ships as targets. 
The analysis of the data for this phase of the study has not been 
completed as of this writing. 

In support of transfer data showing the value of training devices, 
Is the analysis of training situations to determine tasks that can and 
cannot be trained In the training device and In the operational situation. 
An example of such an analysis will be given for Device 14A2, Surface 
ASW Attack Trainer, for which a study of the retention of skills learned 
on It was performed, but for which no transfer of training data are 
available. In the sk^ll retention study (Ref. 23), performance changes 
by members of ASROC teams undergoing training at Norfolk were measured, 
and their skills reevaluated at periods ranging from eight to sixteen 
weeks after training. Two rather straightforward conclusions were 
reached. One Is that the trainees do. In fact, learn In the trainer. 
The other Is that they rapidly forget what they have learned when they 
go to sea. It was concluded that shorebased team training should be 
made a regular part of the operating schedule of ASROC-equlpped ships. 
The consensus expressed was that Device X14A2 practice was as good or 
better than at-sea practice, since It allows for multi-unit problems 
and unexpected contingencies. 

TAEG team members analyzed the tasks that can be performed on 
Device 14 A2 and in the operational situation. They concluded that if 
Device 14E19, Basic Operator/Team Trainer for the AN/SQS-26CX Sonar, 
is used in conjunction with Device 14A2, everything that could be 
trained at sea could be conducted in the trainer. In fact, at sea it 
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is extremely difficult to conduct other than basic training problems, and 
intermediate and advanced level exercises avi^ virtually never conducted. 
Intermediate and advanced exercises are not conducted in the trainer 
either, but such is not only feasible, but highly desirable. This 
analysis could apply in general to various attack trainers. Wlr rher this 
analysis is correct will b** tested soon with the planned traini^vg effective- 
ness evaluation of Device 14A2 which will obtain transfer measurements. 

A recent study conducted by HumRRO (Caro, 1970, Ref . 24) provides 
some support that transfer can be predicted. An approach called Equipment- 
Device Task Commonality Analysis was developed. It identifies the stimuli 
and responses involved in the operational situation and the training 
device, and then determines the extent to which the stimuli are common 
to both the operational equipment and the device* It then looks at 
respons^e commonality, that is, the extent to which responses made in the 
operational environment may be made in the training device. A prediction 
of positive or negative transfer of training from the device to the 
operational equipment is made from the information concerning stimulus 
and response commonality, and using the following '"principles" of transfer 
of training as guidelines: (1) Positive transfer will occur when both 
stimuli and responses are similar in the training situation and the 
operational situation, and (2) Negative transfer will occur when the 
stimuli are similar in the training and operational situations, but the 
responses to the similar stimuli are different. This procedure was applied 
to a training device (Link 1-CA-^l, a fixed wing basic instrument trainer 
modified to a helicopter configuration) whose transfer of training value 
had been previously determined empirically. It was concluded that 
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relatively little task commonality exists between it and the operational 
equipment CTH-13T helicopter). Predo:T)inately negative transfer was 
"predicted" from its use, a "prediction" that was supported by the earlier 
transfer of training study. (It is unfortunate that "prediction" was 
not done prior to the transfer study.) 

The relationship between training effectiveness and cost has been 
discussed by many authors since 195A, when Miller (Ref . 25) introduced 
his now well-known hypothetical relationships among degree of fidelity 
of simulation, transfer of training, and simulator cost. His curves 
depicted an increase in both transfer of training and cost with in- 
creasing degree of fidelity of simulation. The objective is to find 
the optimum point of interaction between fidelity, transfer and cost, or 
in other words, to obtain the highest degree of transfer for the lowest 
possible cost. The implication is that it is necessary to make compro- 
mises between economic and training goals when selecting training media. 

The problem with Miller's relationship is not so much the hypo- 
thetical shapes of the curves, but the implicit assumption that training 
value increases as a function of fidelity of simulation. Undoubtedly, 
increasing fidelity of (engineering) simulation results in increased 
cost. However, Miller's curve which shows an increasing amount of 
transfer with increasing fidelity of simulation is disputable. 

A recent study (Erickson, et al, 1972, Ref. 17) stated, "Only a 
handful of studies have been concerned with the relationship between 
fidelity of simulation and training value. Conflicting results have 
been obtained: in some studies high fidelity simulation produced 

20 



24 



better training, while in others, a lesser degree of fidelity produced 
equally good training. No studies have been reported in which higher 
fidelity is associated with poorer training." The authors then go on 
to recommend high-fidelity simulation. 

But even if low fidtiity simulation resulted only in training 
effectiveness equal to that obtained from high fidelity trainers, the 
obvious cost effectiveness requires consideration of low fidelity of 
simulation for appropriate tasks. Many of the studies demonstrating 
that low fidelity training devices are as effective as high fidelity 
devices or operational equipment have been concerned with procedural 
tasks in which every motion must be performed in sequential order (e.g. 
Grimsley, 1969, Ref. 26; Prophet and Boyd, 1970, Ref. 27). Several 
studies were done using flight simulators differing in their degree of 
fidelity of simulation (Mahler & Bennett, 1949, Ref. 13; Wilcoxon, et a 
1954, Ref. 28; Dougherty, et al, 1957, Ref. 29), and one study with 
five different degrees of fidelity in submarine control (Newton, 1959, 
Kef. 30). (The latter study, however, measured transfer not in an 
operational situation but to the simulator having the highest degree 
of fidelity.) Generally, despite differing degrees of fidelity, there 
was no difference in transfer effect between trainers. It is contended 
by the present report that training effectiveness is more a function 
of the manner in which the trainer is used than of the fidelity of the 
trainer • 

The goal to approach complete duplication of operational equipment 
should not be attempted unless a training situation analysis reveals 
its necessity. It is costly to do sc , and in many instances, is 




unnecessary for effective training. The critical element of training 
is transfer to operational performance of the skills, knowledges, and 
attitudes developed in the training situation. This depends very 
heavily on how the training device is used rather than on how realisti- 
cally the device is designed. 

Maximizing fidelity is a very costly endeavor. By minimizing 
fidel'ity, effective training can be provided with a considerable 
reduction in cost, thus resulting in the savings of resources that could 
then be used elsewhere. 

CONCLUSIONS 

IV: 195A, Gagne' (Ref. 31) stated that there are "a number of studies 
on the *ef f ectiveness* of training devices which are generally charac- 
terized by sound but unstartling conclusions." The findings of studies 
since that time are quite similar, so again his statement could be 
made. But, the view held here represents a different interpretation of 
the findings, namely, that trainees can learn some things about flying 
(or other operational tasks) while they are practicing in training 
devices. Though the research results themselves are not necessarily 
"startling", what is startling is the resistance to substitution of 
some operational training time by training devices. It was not until 
costs of non-revenue training flights for the commercial airlines became 
so tremendous that simulators became considered anything but "supplements" 
to flight training. As a result of this demonstration by the airlines 
of the feasibility and practicality of substitution of flight time by 
practice in training devices, the military should boldly adopt the 
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policy of substitution for the appropriate portions of the flight 
training and other operational trainii-g syllabi. 

Training effectiveness evaluations of training systems have demon- 
strated that learning, retention and transfer occur In situations where 
"exact simulation" is not present. (These examples do not necessarily 
violate the theory of "identical elements," which is an approach of 
analyzing transfer in terms of specific elements common to tasks. All 
transfer effects cannot be related to an analysis of specific stimulus- 
response relationships (Ref, 32).) We may generalize from the examples 
of training effectiveness evaluations that training effectiveness results 
not from attempting to approach identity of task elements, but from 
using a training device in a manner that permits trainees to practice 
the behaviors critical for performance in the operational situation. 
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APPENDIX A 
SUMMARY OF TRAINING SYSTEMS 
EFFECTIVENESS EVALUATIONS* 



* Mr. Joseph A. Puig of the Human 
Factors Laboratory prepared most 
of this Summary. 
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SOME GENERAL CONCLUSIONS 

Experiments reveal that substantial amounts of air time can be 
substituted for by simulator time* 

Most experimental work has been done on simple aircraft and trainers* 

Different kinds of flight tasks have different transfer effects. 

The level of simulation and kind of trainer Importantly Influence 
transfer* 

Careful specification of both trainer and operational tasks Is 
necessary If transfer Is to occur. 

Motion of particular kinds affeccs trainee performance and transfer. 

Adding motion and visual displays Increases fidelity requirements. 
Coupling of these Is a major issue. 

How a device Is used may Influence learning and transfer to a greater 
degree than trainer design. 

Differences between training and operational equipment are necessary 
to exploit training technology. 

A precise specification of tasks and measures of operational transfer 
tasks Is vital to effectiveness evaluation. 



39 



35 



REFERENCES 



Gagne, R.M., Foster, Harriet, & Crowley, Miriam E. The measure- 
ment of transfer of training. NAVTRADEVCEN Tech. Rep. 316-1-1, 
1947. (AD 641582) 

Gagn/, R.M., Foster, Harriet, & Crowley, Miriam E. The measurement 
of transfer of training. Psychol. Bull., 1948, pp. 97-130. 

Murdock, B.B. Transfer designs and formulas. Psychol. Bull., 
1957, pp. 313-326. 

Carter, V.E. Future Undergraduate Pilot Training aystem study. 
Final report: Appendix XV, Training effectiveness analysis. 
USAF Aeronautical System Division Tech. Rep. NOR 70-149, 
1971. (AD 881862) 

Povenmire, H.K., & Roscoe, S.N. An evaluation of ground-based 
flight trainers in routine primary flight training. Human 
Factors Journal, 1971, pp. 109-116. 

Caro, P.W. Transfer of instrument training and the Synthetic 
Flight Training System. Proceedings of the Fifth Naval Training 
Device Center and Industry Conference. Tech. Rep. NAVTRADEVCEN 
IH-206, 1972. (AD 737226) 

Ryan, L-E., Puig, J.A. , Micheli, G.S., & Clarke, J.C. An evaluation 
of the training effectiveness of Device 2F90, TA-4J Operational 
Flight Traiiier: Part I: The B-Stage. NAVTRAEQUIPCEN Tech. Rep. 
IH-207. (In preparation) 

Caro, P.W., Isley, R.N., & Jolley, O.B. The captive helicopter 
as a training device: Experimental evaluation of a concept. 
HumRRO Tech. Rep. 68-9, 1968. 

Flexman, R., Townsend^ J.C, & Ornstein, G.N. Evaluation of a 
contact flight simulator when used in an Air Force primary 
pilot training program: Part I: Over-all effectiveness. Air 
Force Personnel and Training Research Center Tech. Rep. 54-38, 
1954. (AD 53730) 

Payne, T.A., Dougherty, Dora J., Hasler, S.G., Skeen, J.R., 
Brown, R.L., & Williams, A.C. Improving landing performance 
using a contact landing trainer. Tech. Rep. SPECDEVCEN 71-16-11, 
1954. (AD 121200) 

Flexman, R.E., Matheny, W.G., Brown, E«L« Evaluation of the School 
Link and special methods of instruction in a ten-hour private 
pilot flight-training program^ University of Illinois Institute 
of Aviation Aeronautics Bull., No. 8, 1950. 



36 



I 



12. Mahler, W.R*, & Bennett, G.K. Special devices in primary flight 
training; their training and selection value. Tech. Rep« 
SPECDEVCEW 151-1-18, 1949. (ATI 64943) 



r 



13. Mahler, W.R. , & Bennett, G.K. Psychological studies of advanced 

Naval air training: Evaluation of operational flight trainers. 
Tech. Rep. SPECDEVCEN 999-1-1, 1950* (AD 643499) 

14. Houston, R^C. Human factors and airline training. American 

Airlines, Inc., Fort Worth, 1970. 

15. Trans World Airlines. Flight simulator evaluation. TWA Flight 

Operations Training Department, 1969. 



r 



16. Meyer, D.E., Flexman, R«E., VanGundy, E.A., Killian, D.C. , & 

Lanahan, C.J. A study of simulator capabilities in an operational 
training program. USAF Aerospace Medical Research Laboratories 
Tech. Rep. 67-14, 1967. 

17. Erickson, H.W. , Simpson, D.W., & Stark, E.A. Naval Undergraduate 

Pilot Training system study. Tech. Rep. NAVTRADEVCEN 72-C-00A9-1. 
(In press) 

18. Lockheed-California Co. Future Undergraduate Pilot Training (UPT) 

study: Phase II Summary Report, Book 3, USAF Aeronautical Systems 
Division, Tech. Rep. LR 23918, 1971. (AD 882298) 



19. Stewart, W.A. , & Wainstein, E.S. Rand symposium on pilot training 
and the pilot career: Final report. Rand Tech. Rep. R-615-PR, 
1970. 



( 



20. Puig, J. The training effectiveness of Device 3A105, Tracked 

Vehicle Driver Trainer (M48A3) . NAVTRAEQUIPCEN Tech. Reo. (In 
preparation) 

21. Finley, Dorothy L. Training effectiveness evaluation of Naval 

training devices: Part I: A study of the effectiveness of a 
carrier air traffic control center training device. Tech. Rep. 
NAVTRAEQUIPCEN 70-C-0258-1. (In preparation) 

22. Robins, J.E., & Finley, Dorothy L. Training effectiveness evaluation 

of Naval training devices: Part II: A study of the effectiveness 
of Device 2F69B, Weapon System Trainer for the P-3A aircraft. 
Tech. Rep. NAVTRAEQUIPCEN 70-C-0258-2. (In preparation) 

23. Schrenk, L.P. , Daniels, R.W. , & Alden, D,G. Study of long-term skill 

retention. (U) Final report. Tech. Ret). NAVTRADEVCEN 1822-1, 1969 
(AD 503679) (The report is confidential) 



37 



ERIC 



41 



1 



24. Caro, P.W. Equipment-device task connnonallty analysis and transfer 

of training. HumRRO Tech. Rep. 70-7, 1970. 

25. Miller, R.B. Psychological considerations in the design of training 

equipment. USAF WADC Tech. Rep. 54-563, 1954. 

26. Grimsley, D.L. Acquisition, retention, and retraining: Effects of 

high and low fidelity in training devices. HumRRO Tech. Rep. 
69-1, 1969. 

27. Prophet, W.W. , & Boyd, H.A. Device-task fidelity and transfer of 

training: Aircraft cockpit procedures training. HumRRO Tech. 
Rep. 70-10, 1970. 

28. Wilcoxon, H.C., Davy, E., & Webster, J.C. Evaluation of the SNJ 

Operational Flight Trainer. Tech. Rep. SPECDEVCEN 999-2-1, 1954. 
(AD 86988) 

29. Dougherty, Dora J., Houston, R.C., & Nicklas, D.R. Transfer of 

training in flight procedures from selected ground training 
devices to the aircraft. Tech. Rep. NAVTRADEVCEN 71-16-16, 1957. 
(AD 149547) 

30. Newton, J.M. Training effectiveness as a function of simulator 

complexity. Tech. Rep. NAVTRADEVCEN 458-1, 1959. (AD 230996) 

31. Gagn^, R.M. Training devices and simulators: Some research 

issues. Amer. Psychol., 1954, pp. 95-107. 

32. Blaiwes, A.S., & Regan, J.J. An integrated approach to the study 

of learning, retention, and transfer - A key issue in training 
device research and development. Tech. Rep. NAVTRADEVCEN IH-178, 
1970. (AD 712096) 

33. Williams, A.C. , & Flexman, R.E. An evaluation of the Link SNJ 

operational trainer as an aid in contact flight training. Tech. 
Rep. SPECDEVCEN 71-16-5, 1949. (AD 637621) 

34. Knauft, E.B., & Buxton, C.E. An experimental study of the effectiveness 

of various training procedures used with the aerial gunnery training 
devices 3- A- 35 and 3-A-2. NAVTRADEVCEN Tech. Rep. 57-1-1, 1946. 
(AD 639275) 

35. Crook, M.N. Interim report on learning studies with the Mark 18 

Coordination Trainer, Device 3-A-40. NAVTRADEVCEN Tech. Rep. 
58-1-1, 1946. (AD 639035) 



38 



ERIC 



42 



1 
'■) 

) 
) 

3 
3 



I 



I 

I 

t 

i 

^ r 

r 
i ( 
] { 
i (' 
\ ( 

; ( 

: 

\ ( 

I ( 
( 

ERIC 



36. Crook, M.N. Report of results of learning studies with Che 

Mark 18 Coordination Trainer, Device 3-A-40. NAVTRADEVCEN 
Tech. Rep. 58-1-2, 1946. (AD 657A85) 

37. Crook, M.N. Report of recommendations for the Mark 18 Coordination 

Trainer Device 3-A-40. NAVTRADEVCEN Tech. Rep. 58-1-A, 19A6. 
(AD 639037) 

38. Knauft, E.C. , Hamilton, C.E., & Spence, K.W. An experimental 

study of learning on the aerial gunnery training device 3-A-2. 
NAVTRADEVCEN Tech. Rep. 57-1-5, 19A7. (AD 639279) 

39. Gottsdanker, R.M. , & Armington, J.C. Final Report. Ranging 

Tracking Aiming Point Assessor Device 3-E-7. NAVTRADEVCEN 
Tech. Rep. 58-1-5, 19A7. (AD 639038) 

40. Weigandt, J.F. , Bishop, E.W. , & Channel, R.C. A study of the 

utilization and design of the Special Devices Center Mark 18 
Model 6 Gunsight Trainer (Device 3-A-40b prototype). Tech. 
Rep. SPECDEVCEN 1043-00-2, 1953. (AD 656633) 

41. Brown, E. L., Matheny, W. G. , & Flexman, R.E. Evaluation of the 

School Link as an aid in teaching ground reference maneuvers. 
Tech. Rep. SPECDEVCEN 71-16-7, 1950. (AD 657473) 

42. Isley, R.N., Caro, P.W., & Jolley, O.B. Evaluation of synthetic 

instrument flight training in the Officer/Warrant Officer Rotary 
Wing Aviator Course. HumRRO Tech. Rep. 68-14, 1968. 

43. Jeantheau, G.G. , & Andersen, B.G. Training system use and effective- 

ness evaluation. Tech. Rep. NAVTRADEVCEN 1743-1, 1966. (AD 640423) 

44. Ryan, L.E., & Micheli, G.S. Training effectiveness evaluation of 

Device 15E18, Tactical ECM Trainer. Unpublished technical report 
(1971). 

45. Meister, D. , Sullivan, D.J., Thompson, E.A., & Finley, D.L. Training 

effectiveness evaluation of Naval training devices. Part II: A 
study of Device 2F66A (S-2E trainer) effectiveness. Tech. Rep. 
NAVTRADEVCEN 69-C-0322-2, 1971. (AD 732795) 

46. Krumm, R.L, & Buffardi, L. Training effectiveness evaluation of 

Naval training devices. Part I: A study of submarine diving 
trainer effectiveness. Tech. Rep. NAVTRADEVCEN 69-C-0322-1, 
1970. (AD 879712) 



39 

43 



ERIC 



UNCLASSIFIED 



StH'urilv Clasxificalion 



DOCUMENT CONTROL DATA R&D 

$%'curity clus*,Hirotion of titlQt body ut ithstntrt ttttJ ittdcxhtf) ,tnnotution nm.>/ he etttercd tv/trn thv owroll rfpori is eta s s Hied) 



1 ORIGINATING ACTIVITY (Corporate ottthot) 

Training Analysis and Evaluation Group 
Naval Training Equipment Center 
Orlando, Florida 32813 



29. REPORT SECURITY C L A SSI r I r A T |0n 

Unclassified 



2b. GROUP 



3 REPORT TITLE 

Analysis of the Transfer of Training, Substitution, and Fidelity of Simulation 
of Training Equipment 



4. OESCRiPTiVE NOTES (Type of report and ineluaivc dotes) 

Final Report - February 1972 - June 1972 



S- AU J HORiSi (First ttamet middle ittitial, last r%ome) 



Micheli, Gene S., Dr. 



0. REPORT OATE 



1972 



So. CONTRACT OR GRANT NO. 



6. PROJECT NO. 

Work Assignment No. 10k2 



la. TOTAL NO. OF PAGES 

41 



76. NO. OF REFS 

46 



9a. ORIGINATOR'S REPORT NUMBER(S) 



TAEG Report 2 



9b. OTHER REPORT NOISI (Any olhvr numbers ihot may br aaslatcd 
fnts report) 



to. oistribution statement 



This document has been approved for public release and sale; its distribution 
is unlimited. 



tl. SUPPLEMENTARY NOTES 



None 



13. ABSTRACT 



12. SPONSORING MILITARY ACTIVITY 



Training Analysis and Evaluation Group 
Naval Trairdng Equipment Center 
Orlando, Florida 32813 



This report summarizes, evaluates and synthesizes the data on the training value of 
training de-'/ices. The report discusses the issues of substitution of some 
operational training time by training devices and the relationship between training 
effectiveness and cost (fidelity of simulation). 



DD 



FORM 1470 

I NOV 65 I *t / «J 

S/N 0102-014^00 



(PAGE 1) 



44 



UNCLASSIFIED 



Security Classification 



KEY WORDS 



Training Effectiveness 
Cost Effectiveness 
Transfer of Training 
Fidelity of Simulation 
Training Media Selection 
Training Effectiveness Prediction 
Training Equipment Substitution 



